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Summary 


State education budgets have shrunk since the economic recession of 2007-09. During the 
2012/13 school year 35 states provided less funding for education than they had five years 
earlier (Leachman, Albares, Masterson, & Wallace, 2016; Levin et al., 2012; Oliff, Mai, 
& Leachman, 2012). As a result, districts across the country are seeking ways to increase 
their efficiency by using fewer resources while maintaining or even improving education 
outcomes. 

This study examines the expenditure-to-performance ratio, a measure that can be used 
along with other information to examine districts’ use of resources as a proxy measure 
of efficiency. District expenditure-to-performance ratios offer a simple descriptive method 
of simultaneously assessing spending and student outcomes using publicly available data. 
However, expenditure'tO'performance ratios can use different measures of expenditure 
and performance, and research has not always explicitly considered how variability in the 
choice of measures and of district characteristics (such as locale, student enrollment size, 
and student poverty status) can affect outcomes. 

Conducted by Regional Educational Laboratory (REL) Northeast & Islands in partnership 
with the REL Northeast & Islands Northeast Rural Districts Research Alliance, this study 
used state department of education of data to create six expenditure-to-performance ratios, 
each calculated by dividing one of three districtdevel measures of per pupil expenditures by 
one of two districtdevel measures of performance. (For guidance on how to calculate these 
ratios, see this study’s companion report, Ryan, Lavigne, Zweig, & Buffington, 2017). The 
six ratios were then used to rank the 98 sample districts in the example REL Northeast & 
Islands Region state. 

Key findings: 

• The rank of each district varied according to which of the six expenditure-to- 
performance ratios was being considered. A district might rank among the top 25 
districts on one ratio but not on another. 

• In the example state a district’s rank on any ratio was often influenced more by the 
performance measure used — one was a measure of growth in student achievement 
and the other a static measure of student achievement — than by the expenditure 
measure. In other words, districts’ ranks may show more movement when compar- 
ing ratios calculated using different measures of performance than when compar- 
ing ratios using different measures of expenditures. 

• Nearly half (43) of the 98 districts ranked among the top 25 districts on at least 
one ratio, but only 8 districts ranked among the top 25 on all six ratios. 

This study demonstrates how, at least within one state, conclusions about district efficien- 
cy may vary depending on which measures of expenditure and performance are conskL 
ered. These findings provide states and districts with evidence about the extent to which 
any single expenditure-to-performance ratio alone can inform perceptions about district 
efficiency. 
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Why this study? 


State policymakers often want to know which districts are achieving the best student 
outcomes given their level of spending. Districts across the country are seeking ways to 
increase efficiency by maintaining or even improving education outcomes while using 
fewer resources. State education budgets have decreased, on average, since the U.S. eco- 
nomic recession of 2007-09 (Leachman, Albares, Masterson, & Wallace, 2016; Levin 
et al., 2012). During the 2012/13 school year 35 states provided less funding for education 
than they had five years earlier (Oliff, Mai, & Leachman, 2012). 


One tool for comparing how efficiently districts within a state use resources in produc- 
ing student outcomes is the expenditure- to-performance ratio (Boser, 2014). 1 This type 
of analysis has also been used to estimate the potential savings that could be achieved 
through school district consolidation, an issue of particular importance for small and rural 
districts (Jimerson, 2007; Rooney & Augenblick, 2009). 

District expenditure-to-performance ratios allow for the simultaneous consideration of dis- 
trict spending and student outcomes through a simple descriptive method that uses public- 
ly available data. However, research about these types of ratios does not always explicitly 
consider how variability in the measures used to calculate the ratios and variability in 
district characteristics (such as locale, student enrollment size, and student poverty status) 
may lead to different conclusions about a district’s ability to use resources efficiently to 
achieve student outcomes. 


District 

expenditure-to- 
performance 
ratios allow for 
the simultaneous 
consideration 
of district 
spending and 
student outcomes 
through a simple 
descriptive method 
that uses publicly 
available data 


This question was raised by members of the Regional Educational Laboratory Northeast 
& Islands Northeast Rural Districts Research Alliance, who wanted to better understand 
whether the selection of some measures of expenditure and performance over others might 
influence the rankings of districts within a state. Thus, this study investigated how a district’s 
perceived efficiency based on expenditure-to-performance ratios may vary with the measures 
used to calculate the ratios (see box 1 for definitions of key terms). Understanding such dif- 
ferences in how districts are perceived is relevant to both district practice and state policy. 


Box 1. Key terms 
Efficiency 

Efficiency. The maximization of educational outcomes for a given quantity of inputs. 

Enrollment 

Student enrollment. An average of the headcount of students living within district boundaries 
during the 2012/13 school year (based on totals collected by districts on October 1 and April 1). 

Measures of expenditures 

Constructed per pupil expenditures. Total spending on 7 of the 11 expenditure categories 
reported by districts in the example state in 2012/13, divided by student enrollment. This 
measure includes spending on regular instruction, other instruction (including summer school 
and extracurricular instruction), student and staff support, system administration, school 
administration, operations and maintenance, and other expenditures and excludes spending 

(continued) 
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Box 1. Key terms (continued) 


on transportation, special education instruction, career and technical education instruction, 
and debt service and other commitments. This measure reflects interest among Northeast 
Rural District Research Alliance stakeholders in creating an expenditure-to-performance ratio 
that excludes expenditures that vary widely across districts — particularly across districts in 
different locales (for example, rural and urban). 

Instructional per pupil expenditures. Total spending on 4 of the 11 expenditure categories 
reported by districts in the example state in 2012/13, divided by student enrollment. This 
measure includes spending on regular instruction, special education instruction, career and 
technical education instruction, and other instruction (including summer school and extracur- 
ricular instruction) and excludes spending on student and staff support, system administra- 
tion, school administration, transportation, operations and maintenance, debt service and 
other commitments, and other expenditures. 

Total per pupil expenditures. Total spending on all 11 expenditure categories reported by dis- 
tricts in the example state in 2012/13, divided by student enrollment. The 11 expenditure cat- 
egories are regular instruction, special education instruction, career and technical education 
instruction, other instruction (including summer school and extracurricular instruction), student 
and staff support, system administration, school administration, transportation, operations 
and maintenance, debt service and other commitments, and other expenditures. 

Measures of performance 

Median student growth percentile in math. A summary of the median student growth rate 
in math in the district relative to the median growth rate in math in the state. For example, a 
median student growth percentile of 60 indicates that the performance of the median student 
in the district grew as much as or more than that of 60 percent of his or her peers in the state 
who had similar assessment score histories. Median student growth percentile is not an equal 
interval variable. 

Percentage of students scoring proficient or above in math. The percentage of a district’s stu- 
dents scoring proficient or above on the state standardized achievement assessment in math. 
Students are assigned to one of four proficiency categories (below basic, basic, proficient, or 
advanced) based on their assessment score. 

State standardized achievement assessment. A statewide assessment used to measure stu- 
dents’ academic knowledge and skills relative to the grade-level expectations for the state. 

District characteristics 

Locale. Census-defined classification system used to describe a school district’s location. 
Codes are assigned based on the location of the district’s central office and its proximity to an 
urbanized area (U.S. Department of Education, 2012). 

City or suburb. City is a census-defined territory inside an urbanized area and inside a principal 
city that can be small (population less than 100,000), midsize (population less than 250,000 
but greater than or equal to 100,000), or large (population greater than 250,000). Suburb is a 
census-defined territory inside an urbanized area and outside a principal city that can be small 
(population less than 100,000), midsize (population less than 250,000 but greater than or 
equal to 100,000), or large (population greater than 250,000). 


(continued) 
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Box 1. Key terms (continued) 


Town. Census-defined territory inside an urban cluster that is at least 10 miles outside of an 
urbanized area (can also be subcategorized as fringe, distant, or remote). 

Rural. Any district falling within the three census-defined rural locale categories: rural fringe 
(rural territory that is less than or equal to 5 miles from an urbanized area or less than or equal 
to 2.5 miles from an urban cluster), rural distant (rural territory that is more than 5 miles but 
less than or equal to 25 miles from an urbanized area or more than 2.5 miles but less than or 
equal to 10 miles from an urban cluster), or rural remote (rural territory that is more than 25 
miles from an urbanized area or more than 10 miles from an urban cluster). 


District enrollment size. A district with a student enrollment of under 1,000 was considered 
small. 

Student poverty status. The percentage of students in the district who are eligible for the 
federal school lunch program (a proxy for low-income status) as reported in the state data 
for 2012/13 and grouped in three categories: below 40 percent, 40-60 percent, and above 
60 percent. 

Source: Houck, Rolle, & He, 2010; Phan & Glander, 2008. 


What the study examined 

The following research question guided this study: 

• Do rankings of a state’s districts based on expenditure-to-performance ratios vary 
when different measures of district expenditures or of district student performance 
are used to calculate the ratios? If so, how do they vary? 


Understanding 
how a district's 
perceived 
efficiency based 
on expenditure-to- 
performance ratios 
may vary with the 
measures used to 
calculate the ratios 
is relevant to both 
district practice 
and state policy 


Six ratios were calculated using expenditure and performance data for the 2012/13 school 
year for each of the 98 K— 12 districts with complete data in an example state in the REL 
Northeast & Islands Region. These six ratios were created from three measures of expen- 
ditures per pupil based on district enrollment 2 and two measures of performance (see box 
2 for a summary of the study data and methods and appendix A for a detailed description). 
This study’s companion guide offers additional guidance on how to calculate these ratios 
(Ryan et al., 2017). 


Three measures of expenditures 

The example state for this study reports 11 expenditure categories overall for each district: 
regular instruction, special education instruction, career and technical education instruc- 
tion, other instruction (including summer school and extracurricular instruction), student 
and staff support, system administration, school administration, transportation, operations 
and maintenance, debt service and other commitments, and other expenditures. These 
11 state expenditure categories were used to calculate three measures of expenditures per 
pupil. 

Two of the measures are commonly used to examine district spending: 

• Total per pupil expenditures, which includes all 11 expenditure categories, in 
dollars. 
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Box 2. Data and methods 


The study used publicly available data from the state department of education’s data ware- 
house on expenditures, performance, and district characteristics from K-12 districts in a state 
in the Regional Educational Laboratory Northeast & Islands Region for the 2012/13 school 
year. District locale data came from the National Center for Education Statistics’ Common 
Core of Data for the 2011/12 school year, the most recent year available at the time of the 
analysis (U.S. Department of Education, 2012). Data on per pupil expenditures were gathered 
directly from the state data warehouse. 

Data on expenditures, performance, and district characteristics were merged, based on a 
unique district identification number, into one dataset to generate descriptive statistics (see 
appendix A for further details about data processing; see appendix B for more information on 
the characteristics of the sample). 

The percentage of district students scoring proficient or above on the state achievement 
assessment was calculated as the number of students who scored proficient or above divided 
by the number of students who took the assessment. 

The study used ratios expressed as units of expenditures to performance outcomes. For 
each district each of the three measures of expenditures was divided by each of the two mea- 
sures of performance, producing six expenditure-to-performance ratios. Districts were then 
rank ordered for each of the six ratios, where a rank of one is assigned to the highest ranked 
district. This report focuses on comparing the top 25 districts on each of the six rankings. A 
sensitivity analysis examined whether findings were sensitive to use of a smaller (top 20) or 
larger (top 30) set of highest ranking districts, but results are consistent regardless of which 
set is used (see appendix A). 


• Instructional per pupil expenditures, which represents spending directly related to 
regular instruction, special education instruction, career and technical education 
instruction, and other instruction, in dollars. 

The third measure was constructed after guidance from the Northeast Rural Districts 
Research Alliance to represent expenditures that alliance members expected would vary 
less by locale (city, town, or rural), enrollment size, or students’ poverty status: 

• Constructed per pupil expenditures, which represents total expenditures excluding 
four types that tend to be higher in rural than in urban or suburban districts: 
special education instructional expenditures, transportation expenditures, techni- 
cal education expenditures, and debt service expenditures, in dollars (Silvernail, 
2006; Killeen & Sipple, 2000; Terman & Behrman, 1997). 


Expenditure-to- 
performance ratios 
may have different 
implications 
depending on 
which expenditure 
measure is used 


Expenditure-to-performance ratios may have different implications depending on which 
expenditure measure is used. Ratios based on total per pupil expenditures reflect how much 
districts are spending in total to produce one unit of performance outcome. Ratios based 
on instructional per pupil expenditures address expenditure over which districts may have 
more control than they do over total expenditures (which include relatively fixed expen- 
ditures for transportation and special education instruction; Ladd, Chalk, and Hansen, 
1999). Ratios based on constructed per pupil expenditures reflect the amount of district 
spending that may be less related to district locale (stakeholders in other locations may 
not find these two ratios useful under their own circumstances if the types of expenditures 
included in the constructed expenditure measure tend to vary less by district characteristics 
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than they do in the example state). By examining the three measures of expenditures, this 
study tests whether states and districts should consider the implications of using different 
types of expenditure measures when calculating expenditure-to-performance ratios. 

Two measures of performance 

The study used two performance measures to calculate expenditure'tO'performance ratios: 

• Median student growth percentile in math. 

• Percentage of a district’s students scoring proficient or above in math. 

The measures were chosen from the publicly available measures of student performance used 
for state accountability purposes and in response to Northeast Rural Districts Research Alli- 
ance member input. These two performance measures were used to identify what difference 
in district ranking on the expenditure-to-performance ratio would result from using a growth 
measure or a proficiency measure (see box 1 for a more extensive definition and appendix A 
for a detailed description). To limit the number of ratios analyzed, the study used measures for 
only one subject, math. Math was selected because research using models of educator effec- 
tiveness suggest that math assessments may lead to more reliable measures of growth than 
assessments for other subject areas (Measures of Effective Teaching Project, 2012). 

District characteristics 

Three district characteristics were examined: 

• Locale (city/suburb, town, or rural). 

• Student enrollment size. A district with student enrollment under 1,000 was con- 
sidered small. 

• Student poverty status, which is the percentage of students eligible for the federal 
school lunch program (a proxy for low-income status). Districts were grouped into 
three categories: below 40 percent, 40-60 percent, and above 60 percent. 

District locale, student enrollment size, and student poverty status were chosen as district 
characteristics of interest because districts that are more rural, smaller in size, or with more 
poverty are commonly perceived by researchers and by policymakers as less efficient (Baker 
& Weber, 2016; Sipple, 2011). To test this perception, the study looked at whether districts 
in rural locales, districts with small student enrollment, and districts with high student 
poverty are systematically absent from among the most efficient districts in the state across 
all six expenditure-to-performance ratios. 

Interpreting the expenditure-to-performance ratios 

Each ratio is interpreted to mean that X dollars in spending are associated with a unit gain in 
the performance measure (see table 1 for all six calculations; see table B2 in appendix B for 
medians of each expenditure-to-performance ratio). As an example, for ratio A assume that a 
district has a total per pupil expenditure of $10,000 and a median student growth percentile 
of 50. The expenditure-to-performance ratio would then be $200 ($10,000/50), meaning that 
$200 in total per pupil spending is associated with a one-percentile point increase in the 
district’s median student growth percentile. These ratios do not indicate cause and effect, but 
they provide a basis for comparing how conclusions about the relative efficiency of districts 
in a state may vary depending on the measures of expenditure and performance employed. 


The two 
performance 
measures were 
used to identify 
what difference in 
district ranking on 
the expenditure- 
to-performance 
ratio would result 
from using a 
growth measure 
or a proficiency 
measure 
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Table 1. Calculation of six expenditure-to-performance ratios 

Ratio 

label 

Expenditure measure 

Performance measure 

Ratio calculation 

A 

Total per pupil 

Median student growth percentile 

Total per pupil expenditures (dollars) 


expenditures (dollars) 

in math 

Median student growth percentile in math 

B 

Total per pupil 
expenditures (dollars) 

Percentage of district students 
scoring proficient or above in math 

Total per pupil expenditures (dollars) 
Percentage of district students scoring 
proficient or above in math 

C 

Instructional per pupil 

Median student growth percentile 

Instructional per pupil expenditures (dollars) 


expenditures (dollars) 

in math 

Median student growth percentile in math 

D 

Instructional per pupil 

Percentage of district students 

Instructional per pupil expenditures (dollars) 


expenditures (dollars) 

scoring proficient or above in math 

Percentage of district students scoring 
proficient or above in math 

E 

Constructed per pupil 
expenditures (dollars) 

Median student growth percentile 

in math 

Constructed per pupil expenditures (dollars) 
Median student growth percentile in math 

F 

Constructed per pupil 
expenditures (dollars) 

Percentage of district students 
scoring proficient or above in math 

Constructed per pupil expenditures (dollars) 
Percentage of district students scoring 
proficient or above in math 

Note: See box 1 and appendix A for description of expenditure and performance measures. 

Source: Authors' compilation. 




The districts were then rank ordered on each of the six ratios, and the correlations between 
district rankings on the six ratios were calculated. A rank correlation measures the statis- 
tical dependence between two district rankings. A perfect rank correlation of 1 indicates 
that the rank-order of districts is identical across the two ratios being compared. 

What the study found 


Each of the six expenditure-to-performance ratios resulted in a different ranking of dis- 
tricts. The rankings of these 98 districts were influenced more by the measures of perfor- 
mance than by the measures of expenditures. Only 8 of the 98 districts were ranked in 
the top 25 districts on all six ratios, and those 8 did not vary systematically from the other 
districts in characteristics related to locale, enrollment size, or student poverty status. 


The six 

expenditure-to- 
performance 
ratios analyzed 
were positively 
but not perfectly 
correlated, and 
each ratio returned 
a different set of 
district rankings 


The six expenditure-to-performance ratios were not perfectly correlated, and each yielded a 
different ranking of districts 

The six expenditure-to-performance ratios were positively but not perfectly correlated, 
and each ratio returned a different set of district rankings. It was hypothesized that ratios 
sharing a numerator (measure of expenditures) or a denominator (measure of performance) 
would be strongly, although not perfectly, correlated (that is, a correlation > .5 but less 
than 1; Cohen, 1988). As anticipated, the correlations between ratios were strong and 
positive, ranging from .47 to .93 (table 2). Nevertheless, these ratios were not perfectly 
correlated, so each provides somewhat different information about district expenditures as 
they relate to performance and yields a different ranking of districts. 
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In the example state the measure of performance influences districts’ rankings more than the 
measure of expenditures does 

The correlations between ratios that shared the measure of performance (denominator) 
were stronger than the correlations between ratios that shared the measure of expendi- 
tures (numerator). Ratios that share a measure of performance but have different measures 
of expenditures were highly correlated (coefficients from .79 to .91; see table 2). Ratios that 
have different measures of performance but the same measure of expenditures were more 
weakly correlated (coefficients from .61 to .65). So in this state, the choice of performance 
measure had a stronger influence on a district’s rank than did the choice of expenditure 
measure. 

Although nearly half of the districts examined ranked in the top 25 districts on at least one ratio, 
only 8 districts consistently ranked in the top 25 districts on all six ratios 

To illustrate how different expenditure-to-performance ratios could lead to different con- 
clusions about district efficiency, the study team examined how many districts that ranked 
among the top 25 on at least one ratio did so on all six ratios. Of the 98 districts 43 were 
ranked in the top 25 on at least one ratio, but only 8 were ranked in the top 25 on all six 
ratios (figure 1). 

The eight districts ranking high on all six ratios differed in district characteristics (locale, 
enrollment, and poverty status), but they did not appear to vary systematically from the 
other districts in the study. Five of the eight districts were rural, with less than 60 percent 
of students eligible for the federal school lunch program (table 3). Only one of the eight 
districts had more than 60 percent of students eligible for the federal school lunch program 
(see appendix table B1 for the total number of districts in each locale, student enrollment 


Table 2. Rank correlations for six expenditure-to-performance ratios, 2012/13 

Ratio 

label 

Expenditure to performance ratio Ratio A Ratio B 

Ratio C 

Ratio D 

Ratio E 

Ratio F 

A 

Total per pupil expenditures (dollars)/ 

Median student growth percentile in math .61 

.84 

.58 

.91 

.59 

B 

Total per pupil expenditures (dollars)/ 

Percentage of district students scoring proficient 

or above in math 

.47 

.90 

.54 

.93 

C 

Instructional per pupil expenditures (dollars)/ 

Median student growth percentile in math 


.63 

.79 

47 

D 

Instructional per pupil expenditures (dollars)/ 

Percentage of district students scoring proficient 
or above in math 



.54 

.87 

E 

Constructed per pupil expenditures (dollars)/ 

Median student growth percentile in math 




.65 

F 

Constructed per pupil expenditures (dollars)/ 

Percentage of district students scoring proficient 
or above in math 






Note: Bolded green correlation coefficients share a common numerator. Bolded black correlation coefficients share a common denom 
inator. Correlation coefficients are based on Spearman rank correlations. Tests of significance are not reported because correlations 
were calculated using the population of 98 K-12 districts with complete data. 

Source: Authors' analysis of 2012/13 data for 98 K-12 districts with complete data from the website of the department of education 
of a state in the Regional Educational Laboratories Northeast & Islands Region. 


In the example 
state the choice 
of performance 
measure had a 
stronger influence 
on a district’s rank 
than did the choice 
of expenditure 
measure 


7 




Figure 1. Forty-three districts were ranked in the top 25 districts on at least one expenditure-to 
performance ratio, but only 8 were ranked in the top 25 on all six ratios, 2012/13 


Total number of districts 



Source: Authors' analysis of 2012/13 data for 98 K-12 districts with complete data. Data on expenditure and performance, enroll- 
ment, and eligibility for the federal school lunch program from the website of the department of education of a state in the Regional 
Educational Laboratory Northeast & Islands Region; data on locale from U.S. Department of Education (2012). 


Table 3. Characteristics of districts in the top 25 districts on all six expenditure-to- 
performance ratios, 2012/13 

District 

Average rank 
over six ratios 

Locale 

Student enrollment 

Poverty status 
(percentage of students 
eligible for the federal 
school lunch program) 

1 

6 

Town 

2,000+ 

40-60 

2 

7 

Rural 

250-999 

< 40 

3 

9 

Town 

2000+ 

> 60 

4 

13 

Rural 

1,000-1,999 

< 40 

5 

15 

Rural 

2,000+ 

< 40 

6 

15 

City or suburb 

2,000+ 

40-60 

7 

17 

Rural 

250-999 

40-60 

8 

21 

Rural 

2,000+ 

40-60 


Source: Authors’ analysis of 2012/13 data for 98 K-12 districts with complete data. Data on expenditure and 
performance, enrollment, and eligibility for the federal school lunch program from the website of the depart- 
ment of education of a state in the Regional Educational Laboratory Northeast & Islands Region; data on 
locale from U.S. Department of Education (2012). 


8 

















size, and student poverty status category). The eight districts also varied in student enroll' 
ment, although none had enrollment below 250 students. Nevertheless, although the eight 
districts varied in the characteristics for which data were publicly available, it is possible 
that the eight districts varied systematically from other districts in ways that were not 
captured by this study. 


Implications of the study findings 


Conclusions about how efficiently districts use expenditures in relation to performance 
will likely vary depending on which measures of expenditure and performance are used. 
A common question from policymakers is “Which districts in the state are using resources 
most efficiently?” However, this study’s findings suggest that a more appropriate question 
may be “Which districts are using specific types of resources (expenditures) most efficiently 
to produce particular student outcomes (performance)?” In other words, state policymakers 
may want to select the measures of expenditure and performance that are most relevant to 
the issues they are interested in. 

Two implications of this study stand out: 

• The selection of performance measures may be especially important and could 
range from scores on achievement assessments (as used in this study) to school 
attendance and high school graduation rates. 

• Descriptive evidence indicates that the 8 districts that appeared in the top 25 
districts on all six expenditure-to-performance ratios varied in locale, enrollment 
size, and student poverty status. These 8 districts included one or more districts 
that were rural in locale, had a small enrollment size, or had a high percentage of 
student poverty. Thus, the most efficient districts do not appear to be determined 
solely by any of the district characteristics examined in this study. This study 
demonstrates that rural, small enrollment, and high-poverty districts do rank 
among the most efficient districts in their state, regardless of the expenditure- 
to-performance ratio that is used. So in addition to examining district rankings, 
it is equally important to look at what districts of all kinds are doing to achieve 
efficiency. Recent research has begun to examine how small, rural districts may 
be improving efficiency and success through such practices as community col- 
laboration, creative leadership, and frugality (Roza & Heyward, 2015). Addition- 
al research, including site visits and interviews, might be helpful in discovering 
other practices related to resource use that are unique to these consistently high 
ranking districts. 


The eight districts 
ranking high 
on all six ratios 
differed in district 
characteristics 
(locale, enrollment, 
and poverty status), 
but they did not 
appear to vary 
systematically from 
the other districts 
in the study 


To the extent that differences in district rankings appear to reflect contextual differences 
rather than differences in how districts are spending resources, it may be misleading to 
draw conclusions on the basis of any single ratio. Particularly when the results will be 
used for high-stakes decisionmaking, states should move beyond this type of descriptive 
analysis to analyses that statistically account for contextual differences among districts. 
Some states already create district reference or benchmark groups based on student and 
district characteristics. Comparing expenditure-to-performance ratios of districts within 
these groups could help determine how efficiently districts are spending resources. 
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Limitations of the study 


This study has three main limitations. 

• The districts included in the study were limited to K-12 districts, and the study 
relied on publically available data from a department of education website in a 
state in the REL Northeast & Islands Region and data from the Common Core 
of Data (U.S. Department of Education, 2012). Because expenditures tend to vary 
depending on the school district type (K-8 versus K-12), the study team did not 
include all districts from the example state and instead focused on K-12 districts. 
While this study may help state and district leaders think about which measures of 
expenditure and performance are most relevant for policy decisions and consider 
whether to explore how the use of different data affects the results, it does not 
represent an analysis of complete data from the example state and thus should not 
be seen as an assessment of statewide efficiency. 

• The analyses reported here provide states and districts with an example of a 
straightforward approach to calculating expenditure-to-performance ratios and 
comparing the results using a ranking procedure. However, there are many sys- 
tematic approaches to analyzing expenditure and performance (for example, see 
Lafortune, Rothstein, & Schanzenbach, 2016). If states or districts have the goal of 
using estimates for decisionmaking, they might consider using more formal mod- 
eling approaches and evaluating the sensitivity of the estimates obtained through 
those analyses before using them as a basis for high-stakes decisions (see Board- 
man, Greenberg, Vining, and Weimer, 2011; Hanushek, 2006; and Levin et al., 
2012 for additional information). 

• The analyses for this study are descriptive and cannot disentangle the influences of 
the use of different expenditure and performance measures from the district char- 
acteristics that were examined. Thus, causal conclusions cannot be drawn from 
this work. It should not be inferred that a district’s expenditure-to-performance 
ranking is attributable to the components of any one ratio calculation or to one 
district characteristic (for example, that some districts can spend less for the same 
student outcomes because of their locale). However, these descriptive analyses 
illustrate the potential relevance of different measures in addressing research ques- 
tions of interest. This type of analysis also shows states how they may use existing 
data to compare the ranking of districts on different expenditure-to-performance 
ratios. 


The analyses for 
this study are 
descriptive; thus, 
causal conclusions 
cannot be drawn 
from this work 
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Appendix A. Data and methodology 


This appendix describes the data and methodology used to address the research question. 

Study population 

The study included all 98 K-12 school districts in a state in the Regional Educational 
Laboratory Northeast & Islands Region for which expenditure and performance data was 
available from the state’s department of education. 

Data sources and measures 

All data used in the study were publicly available from the state’s department of education 
and the National Center for Education Statistics’ Common Core of Data (U.S. Department 
of Education, 2012). Researchers used a district identification number as a unique identifier 
to merge data from the department of education and the Common Core of Data. To ensure 
that the merge had succeeded, the resulting data set was examined by district name. 

Measures of expenditures. Per pupil expenditures by district enrollment for 2012/13 came 
from the state’s education website, which reports per pupil expenditures by budget cat- 
egory. Eleven categories of expenditures are reported: regular instruction, special educa- 
tion instruction, career and technical education instruction, other instruction (including 
summer school and extracurricular instruction), student and staff support, system admin- 
istration, school administration, transportation, operations and maintenance, debt service 
and other commitments, and other expenditures. 

Researchers considered three expenditure measures based on the categories available: total 
per pupil expenditures, instructional per pupil expenditures, and constructed per pupil 
expenditures, each based on district enrollment. Total per pupil expenditures is equal 
to the sum of per pupil expenditures across all 11 reported categories for each district. 
Instructional per pupil expenditures refers to the categories related to instruction: regular 
instruction, special education instruction, career and technical education instruction, 
and other instruction. Constructed per pupil expenditures is the sum of expenditures for 
regular instruction, other instruction, student and staff support, system administration, 
school administration, operations and maintenance, and other expenditures based on the 
number of students within the district, excluding expenditures associated with transporta- 
tion, special education, technical education, and debt service — costs that have been found 
to vary according to locale (Terman & Behrman, 1997). 

Measures of performance. The state-based assessment system provides measures of growth 
and proficiency that the study used as performance measures: median student growth per- 
centile in math for grades 4-8 and the percentage of students in grades 3-8 who scored 
proficient or above on the state math achievement assessment. Both performance mea- 
sures were for 2012/13. The measures were chosen based on publicly available measures of 
student performance used for state accountability purposes and in response to Northeast 
Rural Districts Research Alliance member input. This study included both performance 
measures in order to identify any descriptive differences in rankings of districts that might 
emerge from the use of a growth measure compared with the use of a proficiency measure 
in calculating expenditure-to-performance ratios. The study used performance measures 
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for only one subject, math, to limit the number of ratios analyzed to six; including a second 
subject would have doubled the number of ratios. Math was selected rather than literacy 
because research from models of educator effectiveness suggests that math assessments may 
lead to more reliable measures of growth (Measures of Effective Teaching Project, 2012). 

Student growth percentiles are based on performance over two consecutive years and 
are reported for each school and district in math and reading beginning with grade 4. 
A student growth percentile represents how much a student improved compared with a 
particular group of students with the same initial score. For example, a student growth 
percentile of 60 indicates that the student performed better, on average, than 60 percent of 
peers with the same previous score. The median student growth percentile for each district 
was obtained directly from the state’s department of education website. 

The percentage of students who scored proficient or above in math is equal to 100 times 
the total number of students who scored proficient or above in grades 3-8 divided by the 
total number of students who took the math assessment. 

District characteristics. The study reported expenditure-to-performance ratios according 
to district characteristics including locale, student enrollment size, and student poverty 
status. District locale was categorized based on the Common Core of Data’s 2011/12 locale 
codes, which are consistent with the locale codes used by the U.S. Census Bureau: city or 
suburb, town, rural fringe, rural distant, and rural remote (U.S. Department of Education, 
2012). Rural fringe, rural distant, and rural remote categories were collapsed into a single 
rural locale category (see box 1 in the main report). The number of students in the dis- 
trict was also gathered from the state department of education for 2012/13; the categories 
for enrollment were based on the National Center for Education Statistics categories: less 
than 250, 250-999, 1,000-1,999, 2,000-4,999, and 5,000 or more. The top two categories 
were collapsed into a 2,000+ category because few districts in the state enroll 5,000 or 
more students (U.S. Department of Education, 2012). 

District poverty status was determined based on the percentage of students eligible for the 
federal school lunch program, as reported by the state department of education. Districts 
were categorized into three poverty levels based on percentage of students eligible for the 
federal school lunch program: below 40 percent, 40-60 percent, and above 60 percent. 

Data processing and analysis 

This section describes the analyses that were used to address the research question on 
whether district rankings based on expenditure-to-performance ratios vary when different 
expenditure or student performance measures are used. 

Computing expenditure- to -performance ratios. Each ratio was calculated as one measure of 
expenditures per pupil divided by a measure of performance. To address the research ques- 
tion, three pairs of ratios were computed. The first pair used total per pupil expenditures 
by enrollment, the second pair used instructional per pupil expenditures by enrollment, 
and the third pair used the constructed measure of expenditures per pupil by enrollment. 
Each pair used one of the three measures of expenditures and two measures of performance 
— district median student growth percentile in math and district percentage of students 
scoring proficient or above in math as measured by the state achievement assessment. 
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After the six ratios were calculated for districts with complete expenditure and perfor- 
mance data, districts were ranked based on each ratio (N = 98), and Spearman’s rank 
correlation coefficients were calculated for the relationships among the six ratios, which 
resulted in 15 correlations (see table 2 in the main report). Finally, the study team identi- 
fied the extent to which the same districts appeared among the 25 highest ranking districts 
ranked by each of the six expenditure-to-performance ratios. 

Conducting sensitivity analysis. To examine the sensitivity of the findings to the use of 
25 as the cutoff for identifying the highest ranking districts, the study team conducted two 
additional sets of rank-order analyses, producing a top 20 and top 30 for each of the six 
expenditure-to-performance ratios. Characteristics of the schools — rural/not rural, enroll- 
ment under/over 1,000, and more/less than 60 percent of students eligible for the federal 
school lunch program — were consistent regardless of whether a top 20, top 25, or top 30 
was considered (table Al). There were no instances in which the discrepancies between 
the top 25 and top 20 or top 30 was greater than 10 percentage points. There were only 
four instances where the discrepancies between the top 20 and top 30 were greater than 
10 percentage points. 


Table Al. Sensitivity analysis reporting the percentage of districts with particular characteristics that 
appear among the top 20, 25 , and 30 districts based on each expenditure-to-performance ratio in the 
example state in the Regional Educational Laboratory Northeast & Islands Region, 2012/13 





Rural 


Student 
enrollment 
under 1,000 

Student 
enrollment 
under 250 

Above 60 percent 
of students 
eligible for the 
federal school 
lunch program 

Ratio 

label 

Expenditure to performance ratio 

Top 

20 

Top 

25 

Top 

30 

Top 

20 

Top 

25 

Top 

30 

Top 

20 

Top 

25 

Top 

30 

Top 

20 

Top Top 
25 30 

A 

Total per pupil expenditures (dollars)/ 

Median student growth percentile in math 

50 

56 

53 

30 

24 

23 

5 

4 

3 

25 

24 23 

B 

Total per pupil expenditures (dollars)/ 
Percentage of district students scoring 
proficient or above in math 

60 

60 

57 

25 

28 

23 

0 

4 

3 

25 

24 20 

C 

Instructional per pupil expenditures (dollars)/ 
Median student growth percentile in math 

70 

68 

70 

45 

36 

37 

10 

8 

10 

25 

28 27 

D 

Instructional per pupil expenditures (dollars)/ 
Percentage of district students scoring 
proficient or above in math 

80 

72 

63 

45 

40 

33 

15 

12 

10 

25 

20 17 

E 

Constructed per pupil expenditures (dollars)/ 
Median student growth percentile in math 

50 

60 

60 

15 

20 

27 

5 

8 

7 

25 

24 23 

F 

Constructed per pupil expenditures (dollars)/ 
Percentage of district students scoring 
proficient or above in math 

55 

56 

57 

20 

16 

17 

0 

0 

0 

15 

12 17 


Percentage of districts in the sample 


67 



37 



12 



22 


Source: Authors’ analysis of 2012/13 data for 98 K-12 districts with complete data. Data on expenditure and performance, enroll- 
ment, and eligibility for the federal school lunch program from the website of the department of education of a state in the Regional 
Educational Laboratories Northeast & Islands Region; data on locale from U.S. Department of Education (2012). 
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Appendix B. Supplemental statistical tables and figures 


The tables and figures below summarize the characteristics of districts included in the 
study, describe the variables used to calculate expenditure-to-performance ratios, and 
display how different rankings based on these ratios vary by district characteristics. 


Table Bl. Number of districts by district locale, enrollment size, and student 
poverty status, 2012/13 

District characteristic 


Student enrollment size 


Student poverty status 
(eligibility for the federal 
school lunch program) 

< 250 

250-999 

1,000- 

1,999 

2,000 
or more 

< 40% 

40-60% 

> 60% 

Locale 

All districts 

12 

24 

25 

37 

24 

50 

22 

City and suburb 

0 

0 

7 

9 

10 

5 

1 

Town 

0 

2 

4 

10 

2 

10 

4 

Rural 

12 

22 

14 

18 

12 

35 

17 

Student poverty status (eligibility for the federal school lunch program) 

Less than 40 percent 

2 

2 

10 

10 




40-60 percent 

6 

10 

13 

21 




More than 60 percent 

3 

12 

2 

5 





Note: Two of the 98 districts did not provide information about the percentage of students eligible for the 
federal school lunch program. 

Source: Authors’ analysis of 2012/13 data for 98 K-12 districts with complete data. Data on locale from U.S. 
Department of Education (2012); data on enrollment and eligibility for the federal school lunch program from 
the website of the department of education of a state in the Regional Educational Laboratory Northeast & 
Islands Region. 


B-l 





Table B2. Median expenditure-to-performance ratios by district locale, enrollment size, and student 
poverty status, 2012/13 



Ratio A 

Ratio B 

Ratio C 

Ratio D 

Ratio E 

Ratio F 

District characteristic 

(Total per pupil 
expenditures 
[dollars]/ 
Median student 
growth percentile 
in math) 

(Total per pupil 
expenditures 
[dollars]/ 
Percentage of 
district students 
scoring proficient 
or above in math) 

(Instructional per 
pupil expenditures 
[dollars]/ 
Median student 
growth percentile 
in math) 

(Instructional per 
pupil expenditures 
[dollars]/ 
Percentage of 
district students 
scoring proficient 
or above in math) 

(Constructed per 
pupil expenditures 
[dollars]/ 
Median student 
growth percentile 
in math) 

(Constructed per 
pupil expenditures 
[dollars]/ 
Percentage of 
district students 
scoring proficient 
or above in math) 

All districts (N = 98) 

234.73 

189.48 

137.53 

108.58 

176.90 

148.32 

Locale 

City/suburb 

224.90 

168.68 

137.83 

100.06 

159.98 

118.36 

Town 

227.35 

186.61 

136.63 

111.66 

161.62 

133.03 

Rural 

241.98 

198.49 

137.65 

109.30 

169.05 

136.69 

Student enrollment size 

Fewer than 250 students 







(n = 12 ) 

263.22 

287.16 

133.84 

143.00 

196.91 

210.99 

250-999 students 







(n = 25) 

241.80 

201.96 

137.42 

111.76 

175.57 

142.96 

1,000-1,999 students 
(n = 26) 

237.35 

192.75 

139.75 

107.50 

167.79 

135.00 

2,000 or more students 
(n = 35) 

227.58 

169.17 

136.73 

101.30 

158.26 

118.87 

Student poverty status (eligibility for the federal school lunch program; n = 96) 

Less than 40 percent 

234.10 

168.68 

139.64 

99.96 

161.61 

117.81 

40-60 percent 

243.58 

198.50 

136.52 

110.68 

167.88 

138.06 

Above 60 percent 

244.58 

202.82 

138.13 

125.04 

160.57 

140.59 


Note: Two of the 98 districts did not provide information about the percentage of students eligible for the federal school lunch program. 

Source: Authors' analysis of 2012/13 data for 98 K-12 districts with complete data. Data on expenditure and performance, enroll- 
ment, and eligibility for the federal school lunch program from the website of the department of education of a state in the Regional 
Educational Laboratory Northeast & Islands Region; data on locale from U.S. Department of Education (2012). 
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Notes 


The authors thank members of the Northeast Rural Districts Research Alliance, especially 
members of the Advisory Committee for this study David Silvernail, John Sipple, and 
James Sloan. The Advisory Committee members contributed to the research design and 
reports of this study. 

1. In the education literature measures of efficiency are sometimes referred to as mea- 
sures of cost-effectiveness. However, cost-effectiveness analysis, which is used to assess 
which among two or more interventions shows the highest effectiveness relative to 
cost, typically requires accounting for all costs, both tangible (materials, staff sala- 
ries) and intangible (parent volunteer hours, donations), involved in each intervention 
(Levin & Belfield, 2015). A cost-effectiveness analysis also requires a causal estimate 
of each intervention’s impact on the outcome of interest, as opposed to a descriptive 
outcome measure. The expenditure-to-performance ratios presented in this study do 
not reflect any causal association between district spending and student performance. 

2. Some states report two enrollment counts: enrollment based on district of residence 
and enrollment based on district of attendance. When applicable, stakeholders should 
consider calculating expenditure-to-performance ratios using both enrollment counts 
to explore whether conclusions about district efficiency vary depending on the enroll- 
ment count used. For the purposes of this analysis, resident enrollment was used to 
calculate ratios with total and instructional expenditures, and the state’s attending 
enrollment was used to calculate ratios with the constructed expenditure measure. 
Correlations were examined between ratios estimated using expenditures for enroll- 
ment based on district of residence and enrollment based on district of attendance, and 
the results suggested that this enrollment distinction, while tracked in this example 
state, would not influence the current set of findings. 
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Making Connections 

Studies of correlational relationships 


Making an Impact 

Studies of cause and effect 


What’s Happening 

Descriptions of policies, programs, implementation status, or data trends 


What’s Known 

Summaries of previous research 


Stated Briefly 

Summaries of research findings for specific audiences 


Applied Research Methods 

Research methods for educational settings 


Tools 

Help for planning, gathering, analyzing, or reporting data or research 





